Be ExPeRT (Behavioral Health Expansion in Pediatric Residency Training): A Case-Based Seminar

Introduction Pediatric residents report behavioral or mental health (B/MH) assessment and treatment as a training gap and often feel ill-equipped to address these issues in clinical practice. We developed a novel interactive training program to improve resident confidence in managing common pediatric B/MH conditions. Methods The Be ExPeRT curriculum comprised a half-day interactive seminar on attention deficit hyperactivity disorder, anxiety, depression, and suicidality followed by monthly case-based discussions. Content included didactic material, role-play, and case discussion. The training was optional and open to pediatric or combined medicine-pediatrics trainees. Results Twenty-three residents (70% female) participated in four separate seminars over 2 years. Of the participants attending the seminars, 17 (74%) completed the presurvey, and 16 (70%) completed the postsurvey. Statistically significant improvement was noted in comfort treating major depressive disorder (41% pre, 94% post, p = .002), suicide risk (29% pre, 94% post, p < .001), and anxiety (24% pre, 94% post, p < .001) following program participation. Twelve (75%) of the 16 participants completing the survey rated the training in the top 5%–10% with respect to other resident learning experiences. Discussion We developed this curriculum to enhance trainee knowledge and comfort in addressing common pediatric B/MH conditions in primary care. Significant improvement was noted in self-reported comfort in treating major depressive disorder, suicide risk, and anxiety, and the program was well received. The curriculum can be adapted for use in any training program for primary care providers to provide B/MH education that may be lacking or supplement existing programming.


Introduction
Almost 20% of children have a behavioral or mental health (B/MH) condition, and half receive no treatment. 1 As many as 70% of primary care visits involve a B/MH complaint, and primary care providers (PCPs) report feeling unprepared to address these issues. 2 The scarcity of B/MH specialists has hindered children and families in accessing treatment despite the increasing demand for it. 3 In fact, only half of referrals from PCPs successfully connect children to B/MH specialists, highlighting the need to at least begin B/MH assessment and treatment in primary care. 4,5 conditions, and pediatric program directors consistently identify B/MH as a training gap. 4,5 Several organizations have recognized this essential need and formally called for intervention. The American Academy of Pediatrics published a policy statement in 2009 that set forth B/MH competencies for primary care pediatricians along with recommendations for achieving them. 6 The policy was updated in 2019 to incorporate new data on social determinants of health, trauma, and B/MH management in subspecialty settings. 7 The American Board of Pediatrics also called for improved B/MH education and labeled the training gap as a crisis in 2017. 8 Despite a nationally recognized, urgent need for enhanced B/MH education, widespread change has not been implemented, and the current literature lacks consensus on best practices for doing so. [8][9][10] While the importance of suicide screening is well established, few resources exist to help trainees develop knowledge and skills related to suicide risk assessment and safety planning in primary care. 11,12 As of April 2022, there were three MedEdPORTAL resources on the implementation of suicide assessment during medical training. One is a team-based learning module for medical students on their psychiatry clerkship rotation. 13 Another is a web-based module designed for practicing PCPs and conducted with nurse practitioner and nurse midwife students. 14 The third resource directly targets residents; however, it was published in 2016 and is now outdated in relation to epidemiological data and validated screening tools. 15 With regard to other common pediatric B/MH conditions, Mian and colleagues described a video-based training for pediatric residents to improve identification and treatment of anxiety disorders in primary care settings. 16 There are also several MedEdPORTAL publications on trauma-informed care directed towards pediatric primary care health professionals as well as a resident curriculum on primary care of gender and sexual minority patients. [17][18][19] While these are all highly valuable, none deliver comprehensive content to enhance overall pediatric B/MH competencies and prepare graduating trainees to meet the needs of their patients. Our aim was to develop and pilot an evidenced-based training curriculum for residents to enhance their confidence and knowledge in addressing the most common pediatric M/BH concerns (attention deficit hyperactivity disorder [ADHD], anxiety, depression, and suicidality) that could be expanded to other programs that train PCPs. 20,21 The purpose of this educational intervention is to provide a standardized, evidence-based curriculum built on the principles of adult learning theory that addresses the current educational gap in pediatric resident B/MH training. The target audience is pediatric trainees, including categorical trainees as well as medicine-pediatric combined trainees and pediatric-neurology combined trainees. This curriculum could be expanded to other educational programs with graduating PCPs such as nurse practitioner or physician assistant training programs.

Curriculum Development
The Be ExPeRT (Behavioral Health Expansion in Pediatric Residency Training) program comprised a half-day interactive seminar followed by monthly case-based discussions. The approach was based on Kolb's learning theory of adult pedagogy and emphasized interactive education through experiential learning, reflection, and active application of new material. 22 The monthly case discussions were informed by the Project Extension for Community Healthcare Outcomes method for enhancing professional learning and practice change. 23 We conducted a survey-based needs assessment with participants prior to the development of the curriculum to help inform its content. The seminar didactic materials (PowerPoint presentations, cases, roleplays, and reference guide) were developed by child psychiatry fellows in coordination with a faculty member trained in pediatrics and child and adolescent psychiatry. Cases and role-plays were created to represent typical cases seen in primary care. The seminar was facilitated by child and adolescent psychiatry fellows and faculty. We invited all pediatric residents who had previously completed the course to become cofacilitators for subsequent trainings. We encouraged all facilitators to review the facilitator guide (Appendix A), PowerPoint presentations including references (Appendices B-F), cases, and role-play descriptions prior to program implementation. We held 1-2 hours of preparation sessions with facilitators to review and practice presentations and role-plays. Faculty alternated facilitating the monthly discussion sessions.

Interactive Seminar
The interactive seminar was intended to take 3 hours and covered the topics of ADHD, anxiety, and depression with suicide and safety assessment. At the start of the seminar, a faculty member presented the introductory materials, gave an overview of the course, and reviewed the main learning objectives (Appendix B). We distributed the participant guide (Appendix G), which included validated screening tools, treatment algorithms, dosing guidelines, and nonpharmacologic resources. Residents were encouraged to refer to the guide during the seminar to support active learning. It is recommended that other groups delivering the training modify the slides and participant guide to include resources specific to their location/institution. For each activity in the seminar, the larger group was divided into three smaller groups, with a faculty facilitator in each group.
We began the first topic with a brief PowerPoint presentation outlining the evidence for assessment and treatment of pediatric ADHD, including dispelling common myths about diagnosis and treatment (Appendix C). This was followed by a discussion in small groups where participants applied what they had learned in the lecture to a case. During this small-group activity, participants practiced scoring Vanderbilt ADHD Screening Forms 24 and discussed assessment and treatment of ADHD in a child with school difficulties, including how to engage caregivers in a discussion of the diagnosis and treatment options.
The second hour began with a brief PowerPoint presentation (Appendix D) outlining the evidence for assessment and treatment of pediatric anxiety disorders, including when to treat with therapy, medications, or both. This was followed by a roleplay by faculty members to demonstrate a clinical scenario involving a PCP, caregiver, and child presenting with anxiety, functional abdominal pain, and school avoidance. The second part of the hour involved small groups in which learners acted as the PCP and caregiver and engaged in a discussion about therapeutic and pharmacologic management of anxiety.
The last session taught the principles of assessment and treatment of pediatric depression, with an emphasis on suicide assessment and safety planning (Appendix E). This was followed by a faculty role-play of an adolescent with depression and suicidal thoughts who presented to their PCP with their caregiver. Faculty first demonstrated completing a suicide assessment and safety plan with the patient and caregiver, and then, learners paired up in small groups to role-play suicide assessment and safety planning.
Ten minutes were reserved at the end of the 3-hour seminar for learners to complete the postseminar survey (Appendix H) and sign up to present at a monthly case discussion conference.

Case Discussions
Following the interactive seminar, resident-led case discussion sessions occurred monthly and lasted 45 minutes during protected time for residents, typically at a noon conference. As noted above, upon completion of the initial seminar, residents signed up to present a case at a subsequent monthly case discussion conference. At least one faculty member was present at each monthly case conference to facilitate the discussion and answer questions. We encouraged residents to select a deidentified case from their current caseload that would be relevant to topics presented in the interactive seminar. Prior to each discussion, the presenting resident completed a case discussion worksheet (Appendix I) outlining the basic history of present illness, treatment course, current treatment, and any questions they wanted to discuss about the case. They emailed this form to the learners and facilitators. During the case discussion, the resident presented their case to the group, and the facilitator encouraged the other learners to ask questions and provide suggestions regarding further assessment and treatment. Residents were encouraged to continue attending discussion sessions throughout their residency training (i.e., to join discussion sessions of subsequent Be ExPeRT cohorts).
Conversion From In Vivo to Digital Platform Due to COVID-19 Initially, we held both the interactive seminar and the case discussions entirely in person, with the option of some residents calling into the case discussions if they were offsite for their rotation. Due to social distancing requirements related to the COVID-19 pandemic, we conducted the seminar and subsequent discussion sessions through digital-based conferences via Zoom beginning in March 2020. We utilized features of Zoom to engage participants, using breakout rooms to facilitate group discussion and implementing poll functionality and the group chat features to encourage active participation.

Research Assessment
We asked all participants who attended one of the four Be ExPeRT seminars from January 2019 to November 2020 at an academic medical center in the Southeast to complete surveys pre-and posttraining regarding their perceived knowledge about and comfort with assessment, diagnosis, and treatment of ADHD, anxiety, depression, and suicidality. Surveys developed by REACH (the Resource for Advancing Children's Health) were modified with permission for use with residents. 25 The surveys were administered using the Qualtrics platform. We emailed the presurvey (Appendix J) to all participants prior to the start of the training and gave them time during the half-day seminar to complete the postcourse survey (Appendix H).
This project was determined to be exempt from institutional review board approval. We described pre-and posttraining survey results with frequencies and percentages for the categorical variables. Differences between pre-and posttraining survey results were tested using Fisher's exact test. All analyses were performed using SPSS (IBM SPSS Statistics version 27).

Results
A total of 23 participants attended the four seminars, with approximately four to five participants attending each, except for the first one, which included nine participants. Participation in the monthly case sessions varied, and the mean number of sessions attended was 1.6. Of the 23 residents who participated in the seminars, more than half were pediatrics residents (n = 15), and about one-third were medicine-pediatrics combined residents (n = 8). Nearly three-quarters of participants completed the pre-and postsurveys (presurvey n = 17 [74%], postsurvey n = 16 [70%]).
Results from the pre-and postsurveys are listed in the Table. Prior to completing the Be ExPeRT seminar, nearly threequarters of residents reported feeling knowledgeable about the assessment and diagnosis of each condition taught. A majority of residents indicated that they were comfortable with assessing and diagnosing major depressive disorder (65%), suicide risk (71%), and ADHD (82%), while only about half felt comfortable with assessing and diagnosing anxiety (47%), prior to program participation. Residents' knowledge about and comfort with treatment of the four conditions prior to participating in the seminar varied, with fewer residents reporting knowledge about and comfort with treatment of anxiety (35% and 24%, respectively) compared to the other conditions.
Results indicated that participation in the Be ExPeRT seminar significantly improved residents' knowledge about assessment and diagnosis (71% pre, 100% post, p = .03), as well as knowledge about and comfort with treatment of major depressive disorder (knowledge: 65% pre, 94% post, p = .01; comfort: 41% pre, 94% post, p = .002). Residents' knowledge about and comfort with treating suicide risk significantly improved from pre-to posttraining (knowledge: 53% pre, 94% post, p = .01; comfort: 29% pre, 94% post, p < .001). With regard to anxiety, a significant improvement was found when examining resident's knowledge about and comfort with assessment and diagnosis (knowledge: 71% pre, 100% post, p = .03; comfort: 47% pre, 88% post, p = .02), as well as knowledge about and comfort with treatment (knowledge: 35% pre, 94% post, p < .001; comfort: 24% pre, 94% post, p < .001) of this concern, following participation in the seminar. Notably, a majority of residents reported increased knowledge and comfort managing all domains of major depressive disorder, suicide risk, anxiety, and ADHD at posttraining. Following completion of the seminar, residents also demonstrated significant improvements in their approach to assessing and diagnosing as well as treating suicide risk, with a majority of residents indicating that they would assess and manage patients with a suicide risk (100% and 94%, respectively) rather than refer them to a specialist (Figure). Of the residents who completed the survey, three-quarters (n = 12) rated the training in the top 5%-10% with respect to other resident didactic or learning experiences. Residents' approach to assessing and treating suicide risk. Residents were considered to assess suicide risk if they indicated that they would assess and diagnose all cases, most cases, or only less complicated cases. For treating suicide risk, residents were considered to manage this concern if they indicated that they would manage all cases with no consultation with a mental health colleague, most cases with occasional consultation with a mental health colleague, or only less complicated cases. Residents were considered to refer patients with a suicide risk for assessment or treatment if they indicated they would refer most cases. For both items, the difference in approach from pre-to postsurvey was significant (p < .05). One resident did not complete the item about approach to assessing suicide risk.

Discussion
Effective B/MH education is needed to meet the demands for B/MH services in primary care and to address gaps in training. The implementation of this B/MH training program was found to be feasible in the context of a residency program and effective in imparting information to residents. The Be ExPeRT program was designed to incorporate active participation to increase retention of the material. The program was found to be acceptable to participants and led to improvement in residents' comfort in and knowledge of common B/MH conditions. Of note, a majority of residents rated themselves knowledgeable about and comfortable with the assessment and diagnosis of suicide risk, as well as the assessment, diagnosis, and treatment of ADHD, prior to receiving the training. These ratings may have been influenced by year of training, desirability bias, base rate of ADHD, or comfort of their supervisors in handling ADHD in their clinic settings.
This program has several limitations. Not all participants completed the pre-and postsurveys, which should be considered when interpreting the results, although we did have a fairly high response rate of around 70%. Additionally, the sample size was small, and a majority of participants were female; thus, results may not be generalizable to other programs. The survey questions we included to understand program efficacy are based on residents' perceived behavior and attitudes and do not measure actual practice behaviors. The gold standard for evaluating curriculum implementation is the assessment of practice change (e.g., use of screening tools, initiation of medications to treat B/MH conditions). To support this evaluation, it would be useful to examine participants' B/MH practice behaviors before and after completion of the Be ExPeRT program. Additionally, the opt-in nature of the program may have resulted in self-selection by the most interested and motivated residents. We expect that implementation on a larger scale, for example, mandatory participation within a residency program, would reach participants with lower baseline B/MH knowledge and comfort and result in greater gains following participation.
Despite the self-selected group, attendance at the monthly discussion sessions was suboptimal. Other programs could increase attendance by offering protected time for all trainees to participate and the option to earn a program certificate for attending a predetermined percentage of case discussion sessions. Also, we did not assess whether participation in case discussion sessions or quantity of sessions attended affected participants' comfort with or knowledge of mental health conditions. Another limitation is the potential burden on faculty facilitators. The training has been delivered with a fulltime attending CAP and two child and adolescent psychiatry fellows as well as with two attending CAPs and an attending pediatrician. Allowing fellows or chief residents to teach could facilitate greater feasibility as well as provide an excellent opportunity for trainees to gain valuable teaching skills. It is unclear if the required length or modality of the training is necessary to achieve the reported results or simply if the time spent with child and adolescent psychiatry faculty and fellows affected resident comfort/confidence. Given the demands on faculty facilitators and concerns about sustainability, it will be important to examine potential modifications that could be made to the training, such as shortening its length or delivering content asynchronously, while maintaining its effectiveness. Although participants rated the seminar in the top 5%-10% of teaching activities, it would be beneficial to evaluate fidelity in future seminars to ensure that all instructors are delivering the same content and that participants are gaining similar knowledge and experience.
Future efforts at our institution will aim to increase participation, improve assessment of practice change, and gather additional implementation data to make the curriculum more easily replicable. We also plan to standardize B/MH competency through the use of a skills passport (similar to a procedure log). Other future considerations include developing recorded videos for dissemination as well as increasing the availability of professional development activities for faculty preceptors. This additional work would aim to increase faculty comfort level in supervising B/MH diagnosis and treatment, as this is sometimes a barrier to residents' ability to implement practice change in clinical settings. In addition, while this program was specifically developed for pediatric residents, modified versions have been given to family medicine residents and physician assistant students. The curriculum could also easily be expanded to any group of pediatric primary care clinicians.
In conclusion, there is a compelling need to enhance pediatric trainee education in addressing B/MH issues. Implementation of the Be ExPeRT curriculum improved residents' knowledge of and comfort with treatment of common pediatric B/MH conditions. The curriculum was found to be acceptable and feasible and could be easily generalized to other types of training programs across the country.

Ethical Approval
The Duke University Institutional Review Board deemed further review of this project not necessary.

Disclaimer
The project sponsor played no role in the study design, data collection, analysis, interpretation of data, writing, or revision of the manuscript.